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ABSTRACT 

The paper discusses several of the challenges 
associated with developing a credible reliability 
estimate for a human-rated crew capsule thermal 
protection system. The process of developing such a 
credible estimate is subject to the quantification, 
modeling and propagation of numerous uncertainties 
within a probabilistic analysis. The development of 
specific investment recommendations, to improve the 
reliability prediction, among various potential testing 
and programmatic options is then accomplished 
through Bayesian analysis. 

1. INTRODUCTION 

This paper discusses a Thermal Protection System 
(TPS) reliability estimation process. The process 
utilizes a predictive, probabilistic, physics-based TPS 
model, with uncertainty models informed by various 
contributors, including material properties data and arc 
jet test data. The reliability estimation process 
considers a single TPS failure mode, the bondline over- 
temperature mode, in which the temperature at the 
junction between the TPS material and the underlying 
composite carrier structure is constrained by the carrier 
structure inter-laminar strength requirement. It is a 
notable achievement just to enable quantification of the 
TPS reliability, but the study has moved well beyond 
that accomplishment to identifying and quantifying 
specific uncertainty sources that contribute, quantifying 
the response sensitivities to the mean value and 
dispersion of those uncertainties, and even to making 
specific investment recommendations to improve upon 
the estimated reliability. 

There are many challenges associated with developing 
a credible TPS reliability estimate [1 through 6], In 
this context, the term challenge has several meanings: 
1) identification of the many relevant uncertainties, 2) 
credible quantification of those uncertainties, and 3) 
issues associated with uncertainty reduction and 
mitigation. 


Ten of these challenges are enumerated below and 
discussed subsequently. 

1. Arc Jet Testing 

2. Generic Measured Data Issues 

3. Material Property Effects / Correlations 

4. Trajectory / Orientation 

5. Flow Transition 

6. Ground-to-Flight Uncertainty 

7. Failure Mode Form 

8. Reliability Assessment 

9. Reliability Cascade 

10. Cost / Benefit Modeling 

The reliability assessment process, as shown in Fig. 1, 
is actually the third phase in a TPS synthesis and 
modeling process, all of which depends upon measured 
data from numerous sources, and expert opinion, 
throughout. Color coding has been used in Figs. 1 
through 4: yellow for uncertainty sources, cyan for 
generic process steps, and green for deliverable 
products. 

Details of each of these three phases are illustrated in 
Figs. 2 through 4 and described subsequently. As 
shown in Fig. 2, the code development process 
synthesizes material property data, ablation physics 
data and Subject Matter Expert (SME) opinion and 
results in a fixed, configuration-controlled, ablation 
code version. This fixed code version includes 
embedded code development uncertainties such as 
measured data issues and physics approximations. 
Some of the measured data issues will be described 
subsequently. 

As shown in Fig. 3, the code execution process 
synthesizes one or more vehicle trajectory, orientation, 
aerothermal margining assumption and material 
property specifications and the embedded code 
development uncertainties (noted above) to yield 
reliability case results, which should also be subject to 
configuration control measures. Each of the 
specifications in this process is a synthesis of data and 
SME opinion-based. Additionally, each of the 


reliability case results includes embedded code 
execution uncertainties such as discretization and 
convergence issues. 

As shown in Fig. 4, the reliability assessment process 
synthesizes the results of the code execution process 
through response surface (RS) modelling, subject to an 
RS fit uncertainty, as well as arc jet test reproducibility 
and validation uncertainties, failure mode data and 
modeling uncertainties. 


bondline temperature for the flight TPS based upon the 
test-anchored computations in order to establish the 
TPS reliability. There is considerable uncertainty in 
the arc jet testing and data acquisition process. The 
uncertainty is quantified, assuming normal 
distributions, via the non-dimensional statistical form 
known as the coefficient of variation (COV, given in 
percent, Eq. 1), which is the standard deviation (StDev) 
of an uncertainty distribution, divided by the mean 
value (MV) of the distribution. 


2. THE CHALLENGES 
2.1 Arc Jet Testing 

Arc jet testing is the primary means of obtaining data 
to develop and calibrate the computational models used 
in ablation prediction; a simplified, one-dimensional 
sketch of an arc jet test is shown in Fig. 5. In arc jet 
testing, a TPS sample is exposed to a constant pressure, 
high heat flux plasma stream for time durations of up 
to several minutes. Both convective and radiative heat 
transfer modes are present, but the effects of radiative 
heat transfer have been ignored within this analysis. 
The high heat flux flow quickly increases the surface 
temperature of the material. Thermal conductivity then 
causes the temperature at all points inside the material 
to increase, as well. Typically, thermocouples are 
embedded within the material to measure the thermal 
response as functions of material depth and time. The 
effect of uncertainty in the thennocouple (TC) 
placement, relative to their assumed positions in the 
material, has not been assessed. When testing ablative 
materials, the exposure process causes material on the 
surface to ablate away and the surface undergoes 
recession. Enthalpy effects are also present within the 
arc jet flow. Other phenomena occur such as pyrolysis 
and gas convection through the material. Key material 
properties known to change during the test, such as 
density and thermal conductivity cannot be measured 
during the test. Temperature at bondline, where the 
TPS is joined by adhesive to the underlying carrier 
structure, continues to increase well beyond the 
duration of the test. 

A more realistic depiction of arc jet testing is shown in 
Fig. 6 in which an uneven recession process is 
illustrated. However, the most common state of 
practice, and that used currently by this team, is to 
assume the modelling of the flow is still one- 
dimensional. True atmospheric re-entry situation for 
the TPS is even more complicated than shown in Fig. 6 
since the flow is neither steady nor one -dimensional. 

One specific failure mode is considered within this 
work; this failure mode occurs when the bondline 
temperature exceeds the carrier structure allowable 
temperature. One must attempt to infer the maximum 


cov% = StlDe ^* 100 

MV 


( 1 ) 


Test reproducibility uncertainty of bondline 
thermocouple peak temperatures (comparison of test 
against test at the same conditions in the same facility) 
for the TPS under consideration yields COV values of 
7% (average) to 38% (maximum), among 12 different 
samples for the deepest embedded TC sensors, when 
the mean value and standard deviation are in degrees 
Rankine. Test validation uncertainty of bondline 
thermocouple peak temperatures (comparison of test 
against computation at the same conditions) for the 
TPS under consideration yields COV measures of 8% 
(average) to 45% (maximum), among the 24 different 
samples for the deepest embedded TC sensors. 


2.2 Generic Data Issues 


The overall goal of this work is achieve very high 
reliability with a quantifiable confidence level for the 
TPS and failure mode under consideration; for 
example, a target goal might be stated as to achieve 
99% reliability with 95% confidence. Data issues arise 
in this context simply by the necessary consideration of 
information in the tails of a given uncertainty 
distribution. The target reliability statement implies 
the consideration of data values at two or more 
standard deviations above the mean value of a given 
distribution. As shown in Fig. 7, other significant data 
issues include: bias, model form, dispersion and 
sampling uncertainties (descriptions of each follow). 
Unfortunately, these four types of data issues are 
frequently confounded, and in too many instances, are 
ignored completely. This paper makes an attempt to 
identify and quantify these effects for the problem of 
interest as defined by the following: 

Bias uncertainty is a shift in the mean value behavior 
away from the true, but unknown, mean value behavior 
for the property being sampled. This type of 
uncertainty is typically associated with testing 
techniques and is difficult to identify and account for 
unless tests are conducted in multiple facilities or by 
differing techniques. 


Model form uncertainty is the intentional or 
unknowing use of an incorrect mathematical form to 
describe the behavior being measured. This type of 
uncertainty arises because the true model form of the 
behavior of interest is unknown. Empirical or RS 
modeling techniques are used to develop a modeling 
form for the behavior of interest, but one might attempt 
to model truly cubic behavior with a quadratic model 
form. This type of uncertainty might be identified by 
using data from different sources or by employing a 
robust suite of data modeling options and noting where 
the prediction discrepancies are important. 

Dispersion uncertainty is the use of an approximate 
dispersion model for the behavior of interest, because 
the true dispersion behavior is unknown. The 
uncertainty might manifest itself by the use of a wrong 
distribution type or the by the use of wrong distribution 
parameters (mean value, standard deviation, skewness 
and kurtosis). This type of uncertainty might be 
identified again by using data from different sources or 
by employing a robust suite of uncertainty modeling 
options and noting where the prediction discrepancies 
are important. 

Sampling uncertainty arises through the use of 
statistically small sample sets. This type of uncertainty 
is always present to some extent in most data sets 
because true statistical convergence of the mean value 
and standard deviation of a distribution demands 
million, if not billions, of samples. This type of 
uncertainty can be overcome to some extent through 
the use of k-factors applied to the standard deviation or 
through A- and B-Basis type knockdown factors [7]. 

For the problem of interest, where small data sample 
sets are the norm, sampling uncertainties may amount 
8% (average) to 24% (maximum) of the mean value 
and 5% (average) to 15% (maximum) for the standard 
deviation when considering sets of 100 samples. The 
sampling uncertainties may amount to 2% (average) to 
7% (maximum) of the mean value and 3% (average) to 
5% (maximum) for the standard deviation when 
considering sets of 1000 samples. Even for data sets as 
large as one million samples, the sampling 
uncertainties may amount tenths of a percent for both 
the mean value and standard deviation. This 
magnitude of uncertainty, by itself, would preclude 
fulfilling the target reliability statement. 

The true cost of sampling convergence is usually 
prohibitive. Thus, many engineers and project 
managers are frequently interested in alternative means 
to state the confidence they have in their data. One 
such alternative means is the conventional wisdom that 
one needs at least ten times as many samples as the 
expected probability of failure. Thus, if the expected 
probability of failure is 10% (1 in 10 failures, or 90% 


reliability), that means 100 samples are needed; if the 
expected probability of failure is 5% (1 in 20 failures, 
or 95% reliability), that 200 samples are needed; and if 
the expected probability of failure is 1% (1 in 100 
failures, or 99% reliability), that 1000 samples are 
needed. Still for many applications, this again requires 
a prohibitive number of samples. 

In order to detennine the true minimum number of 
samples required to achieve a given confidence level, 
another means is now proposed. This method, the 
Practical Confidence Limit (PCL), is based upon the 
statistical underpinnings of the ANalysis Of VAriance 
(ANOVA) technique employed within this work. The 
reader is warned that this PCL is a heuristic approach, 
proposed to define the confidence level of a given set 
of data, and is independent from explicit consideration 
of the dispersion of the data set. The PCL will yield 
the bare minimum number of data samples that could 
ever be construed as achieving a given confidence 
level. 

To compute the PCL, one must define the minimum 
number of samples, Nmin, required to construct an RS 
of a given polynomial order (linear, quadratic, etc.) for 
a given number of independent variables or factors. 
This minimum number of samples is defined by 
constructing a D-Optimal Design of Experiment (DOE) 
[8, 9] for the given number of factors and the given 
polynomial order (Norder). That minimum number of 
samples, equal to the number of coefficients needing to 
be evaluated, will define a unique RS model. One 
more sample (Nmin+1, or Nminl) is then required to 
also evaluate the standard deviation of the uncertainty 
function. This minimum number of samples is then 
compared with number of samples currently available, 
N, to detennine the PCL. It is recommended that the 
number of outlier data points (Nout), as defined by 
rigorous statistical tests, be excluded from the 
comparison, since they do not contribute in a 
meaningful way to definition of a credible RS. Thus, 
the PCL is defined as shown in Eq. 2 

PCL% = ( N ~ Nminl ~ Nollt ) ^ i op (2) 

N 

Thus, to achieve a PCL of 95% for a two parameter RS 
(MV and StDev, with Nminl = 2) would require 40 
samples. To achieve a PCL of 99% for a three 
parameter RS (linear model in one factor, plus a StDev, 
with Nminl = 3) would require 300 samples. The 
reader will note the small number of samples required 
compared to the other approaches; moreover, this 
approach is applicable for any definable RS form. 
Table 1 provides the values of Nminl for various 
polynomial orders of RS and various numbers of 
factors. A linear model is considered when Norder = 1 , 


a quadratic when Norder = 2, etc. A two-factor 
interacting (2FI) DOE, in which the cross linear terms 
are included but no higher order terms (quadratic, 
cubic, etc.) is considered when Norder = 1.5; by 
definition, no such model (2FI) is possible if Nfact = 1 
since there is only one factor being considered. For 
any RS model consisting of simply a mean value and 
standard deviation, Nminl =2. 


Table 1. Values of Nminl as a function of the 
polynomial order and number of DOE factors. 


Norder 

Nfact=l 

Mact=2 

N'fact=3 

N'fact=4 

N'fact=5 

1 

3 

4 

5 

6 

7 

1.5 


5 

S 

12 

17 

2 

4 

7 

11 

16 

22 

3 

5 

11 

21 

36 

57 

4 

6 

16 

36 

71 

127 

5 

7 

22 

57 

127 

253 

6 

8 

29 

85 

211 

463 


2.3 Material Property Effects / Correlations 

The problem of interest for TPS depends upon 1 1 
material property characteristics which are input to the 
physics-based ablation code: 

• virgin density of the material (RHOV) 

• char density (RFIOC) 

• virgin thermal conductivity (XKVRG) 

• char thermal conductivity (XKCFIR) 

• virgin specific heat (CPVRG) 

• char specific heat (CPCHR) 

• virgin emittance (EMVT) 

• char emittance (EMCT) 

• normalized recession parameter (ZBPRIM, 
otherwise known as the B’ tables) 

• roughness height (ROUGF1T) 

• pyrolysis gas enthalpy (CPGAS). 

Some of these parameters are measured, other are 
treated as tuning constants in an attempt to calibrate the 
computations to best match arc jet test results. Testing 
for some of these parameters is quite challenging due 
to the environments required (arc jet flows); multiple 
testing techniques may be used for some of the 
material properties. In the present work, a complicated 
situation results that employs data from multiple 
testing sources. Large bias uncertainties are present for 
several variables. Different data sets also used to 


define mean value and dispersion behaviors. The 
accepted values of the material property COV are 
provided in Table 2. However, additional sampling 
uncertainties associated with the mean value and 
standard deviations of the variables, on average 52% 
and 35%, respectively, may be appropriate to consider. 

Table 2. Accepted COV values for the 11 material 
property input variables. 


Variable 

cov% 

RHOV 

4 

RHOC 

4 

CPVRG 

5 

CPCHR 

10 

XKVRG 

7.5 

XKCHR 

25 

EMCT 

2 

EMVT 

5 

ROUGHT 

25 

ZBPRIM 

20 

CPGAS 

25 


An additional concern related to these material 
property variables occurs when the uncertainty of a 
single material property variable simultaneously 
increases both the recession amount and bondline 
temperature. Since increased recession would bring 
the high heat flux flow closer to the bondline, this 
synergistic combination of uncertainty effects can 
potentially lead to a significant under prediction of the 
bondline temperature. This effect does occur (up to 
13% of the bondline temperature in degrees Rankine) 
for several of the key material property variables 
including RHOV, CHCPR, EMCT, ROUGHT and 
CPGAS for the data examined. 

Another material property effect of concern occurs 
when the correlation of two distinct material property 
uncertainties simultaneously increases the recession 
amount and bondline temperature. This synergistic 
combination of uncertainty correlation effects occurs in 
several correlation combinations anticipated by the 
SME team; specially, this situation occurs (up to 20% 
of the bondline temperature in degrees Rankine) for 
correlations between RHOV and RHOC, RHOV and 
CVPRG, RHOV and CPCHR, RHOV and XKVRG, 
RHOV and CPGAS, RHOC and ROUGHT, and for 
RHOC and ZBPRIM. Data bias is also observed in the 
measurements for XKVRG (18%), thermal diffusivity 
(15%), CPVRG (14%) and EMCT (2%) for the data 
examined. These effects are not explicitly considered 
in the present formulation. 


2.4 Trajectory / Orientation 

Uncertainties in the vehicle trajectory during 
atmospheric re-entry may also be significant, due to 
uncertainties in the atmospheric conditions (e.g., 
pressure, density, viscosity) that are experienced. Each 
body point of the heat shield experiences a unique 
combination of pressures and heat fluxes. As shown in 
Fig. 8, seven body points were selected for analysis to 
represent the different TPS sizing regions of the heat 
shield. The colors in Fig. 8 are intended to notionally 
suggest hotter (red) and cooler (blue) surface 
temperature regions of the TPS. The differences 
experienced by the various body points for a given 
mission are considerable: a COV of about 100% in 
recession amount and a COV of about 3% to 11% in 
bondline temperature at specific body points have been 
observed. The situation is exacerbated when the heat 
shield is designed to service more than one mission. In 
this situation, COV increments from one mission to 
another were observed as large as 86% in recession and 
about 40% in bondline temperature. 

2.5 Transitional Flow 

Other effects, such as an uncertain transition front from 
laminar to turbulent flow may significantly affect the 
heat transfer to the TPS. This effect only occurred for 
one of the body points studied. The uncertainty due to 
plausible material property variations, coupled with the 
presence of transitional flow, was found to represent a 
significant increment in both the predicted bondline 
temperature and recession. It is quite possible that the 
a-priori choice of the seven body points does not 
actually identify where the worst conditions occur as a 
result of flow transition. One attempt to identify where 
the worst conditions might occur as a result of flow 
transition was undertaken and resulted in a small 
increment (few percent) to the bondline temperature 
and a larger increment (maybe 20%) in the recession. 

Standard TPS design practice is to assume the 
trajectory uncertainties from mission to mission and 
from body point to body point are covered by design 
margin/ space assumptions (factors of safety, typically 
10% to 15%) applied to the anticipated pressure and 
heat flux. However, these factors of safety provide no 
insight into the true TPS reliability and may even 
provide a false sense of security that TPS is very 
reliable when it is not. Furthermore, this work suggests 
that the established margining assumptions may not 
adequately cover the identified and quantifiable 
uncertainties within the process, especially when 
transitional flow is present. 


2.6 Ground-to-Flight Uncertainty 

Ground-to-flight uncertainty is the uncertainty due to 
TPS testing and development in incorrect aerothermal 
environments, namely ground-based facilities that 
cannot duplicate the three-dimensional continuously 
varying conditions of atmospheric re-entry. It is 
expected that using margining assumptions will cover 
all ground to flight corrections. As this aspect of the 
work was not explicitly addressed within the scope of 
research performed in writing this paper, a literature 
survey was undertaken to identify ground-to -flight 
corrections that others have used [10-13], In particular, 
stagnation pressure corrections of 18% were identified 
by two references, and uncertainties in the heat flux of 
13% to 69% were also found. A code comparison 
conducted by the Thermal Performance Database Team 
yielded thermocouple differences up to 25%. Other 
findings of note were uncertainties in the char 
thickness of up to 42% and uncertainties in the axial 
force coefficient of about 6%, which directly 
influences the ballistic coefficient of the vehicle. An 
investigation of Project Mercury (1961 through 1963) 
revealed potential bondline temperatures uncertainties 
on the order of several hundred degrees [14] which is 
comparable to the current situation. The Apollo 
capsules (1968 through 1972) were designed with 
considerable margin (see Figure 10 in [15]); Apollo 4 
was closest to the design limit. This suggests that there 
has been little advancement in the past 50 years in 
understanding the many critical TPS uncertainties. 

2.7 Failure Mode Form 

This bondline over temperature failure mode is 
illustrated in Fig. 9. Because the exact TPS failure 
conditions are unknown, a model is used to predict 
failure in which the probability of TPS failure increases 
from zero to unity as the bondline temperature 
increases above some safe temperature. However, the 
temperature used to determine the probability of failure 
is subject to numerous uncertainties, as have already 
been noted, and thus is really a distribution with a 
mean value and a standard deviation. Also, the exact 
form of the failure mode is not known and so there 
may be an optimistic representation of the failure mode 
(solid line in Fig. 9), as well as a pessimistic 
representation of the failure mode (dashed line in Fig. 
9); the pessimistic version yields high probabilities of 
TPS failure for a given temperature just above the safe 
temperature. The difference between the two versions 
of the TPS failure probability ( APfail ) has been found 
to be a significant contributor to the reliability 
assessment because just a small change in the failure 
probability (or conversely, the reliability estimate, 
which is Rel = 1 - Pfail) can easily make the difference 


between a fully reliable TPS and one that fails to meet 
the target reliability. 

2.8 Reliability Assessment 

A physics-based probabilistic reliability assessment 
process is assembled and executed. Each of the 
uncertainties identified along the path are modeled 
appropriately. A choice must be made as to how to 
solve the probabilistic reliability assessment problem. 
Typical choices for this are the Monte Carlo 
Simulation (MCS) and First-Order Reliability Method 
(FORM). The advantage of MCS is that is easy to 
implement and execute; however, without a very large 
number of samples, the method is not suitable for very 
high reliability assessments because of the extremely 
slow convergence of properties in the tails of a 
distribution [16]. Millions or billions of physics-based 
model solutions may be required to adequately 
converge the statistical results far away from the mean 
value (i.e., the tails of a distribution) to a high 
confidence level, depending upon the amount of 
dispersion present. Other simulation variants also exist 
including Latin Flypercube and directional methods 
which may require significantly fewer samples to 
achieve comparable convergence levels. The 

advantage of FORM is that it may require significantly 
fewer physics-based model solutions; however, this 
approach is not as robust as MCS and generally 
requires SME oversight. 

Other effects come strongly come into consideration at 
this point, as well. As shown in Fig. 10, the achievable 
reliability is a distribution expected to lie above the 
required or target reliability; this can be easily achieved 
if the required reliability is quite low, such as 50%. 
But, the reliability for mission critical and human 
critical systems, such as the TPS, with no redundancy, 
is typically required to be in excess of 99%. Thus, the 
reliability distribution gets squeezed into a small very 
small window and less tolerance exists for uncertainty 
in the reliability estimate as the required reliability 
approaches 100% (see Fig. 11). 

2.9 Reliability Cascade 

This requirement to achieve very high reliability with 
very little tolerance for uncertainty also propagates 
backward through the system analysis. Moreover, as 
several independent subsystem failure modes feed the 
system failure mode, the required reliability is found to 
be even higher for the subsystems than for the system. 
If the required reliability for the system is defined by 
RSYS, and there are M independent subsystems 
feeding the system failure mode, the required reliability 
for each of the subsystems, RSUB, is given by the 
expression 


RSUB = RSYS 1 ' M (3) 

Thus, if the subsystem reliability for each of two 
independent subsystems is 90%, the system reliability 
is only 81%; conversely, if the required reliability is 
90% for a system with three independent subsystems, 
each subsystem must achieve a reliability of almost 
97%. 

2.10 Cost-Benefit Modeling 

The final challenge, and perhaps the most difficult to 
address in a programmatic sense (so far as achieving an 
overall high confidence level), is that of cost-benefit 
modeling. This challenge must be overcome if one is 
to improve upon the estimated reliability that is 
computed based upon the elements of the previous 
discussion. Each aforementioned uncertainty presents 
a unique investment opportunity. Each uncertainty has 
specific means to reduce the uncertainty and specific 
costs and benefits associated with it as a potential 
investment. In order to make the best specific 
investment recommendations, one must fairly and 
consistently evaluate the cost and potential benefit of 
each investment option. The challenge for this item 
lies in achieving a very high degree of concurrence 
among the various technical and managerial 
stakeholders that the required cost-benefit modeling is 
not only credible but sufficiently correct to enable 
defining valid investment recommendations. 

A sample Bayesian cost-benefit analysis is shown in 
Fig. 12. This process attempts to answer four 
questions: 1) what is the unit of purchase and what 
does a unit of purchase cost, 2) how many units of 
purchase are needed, 3) how much will a unit of 
purchase influence the outcome, and 4) why should 
one invest in this option over another choice? The 
answers to these questions inform the cost-benefit 
modeling process. 

The investment opportunities include obtaining new 
testing results, new testing and measurement 
capabilities and improved computational methods that 
can be brought to bear upon the uncertainty issues. 
Some issues, such as the failure mode modeling might 
be best addressed by convening an SME panel to 
discuss the possible options. It is important to realize 
that the means by which one might reduce one 
uncertainty are very likely quite different from those 
that might address a second uncertainty, as are the 
costs and benefits. 

All of the above information is assembled together into 
a physics-based probabilistic reliability assessment 
process. Each of the uncertainties identified along the 
path must be modelled appropriately (credible 


distribution type and distribution parameters). In this 
work, many of the uncertainties are simply modelled as 
normal distributions, though other distributions could 
and should be investigated to determine the robustness 
of the reliability estimate. The primary outputs of the 
probabilistic reliability assessment process are 
threefold: 1) the mean value reliability estimate, 2) the 
reliability estimate distribution value at 95% 
confidence, and 3) the reliability estimate sensitivities 
to the modelled uncertainties (both mean value and 
standard deviation sensitivities). The first two of these 
three outputs are used to determine if the TPS meets 
the target reliability statement. The third output is used 
to inform other subsequent analyses, such as a resource 
allocation process and a design under uncertainty 
process. 

A resource allocation process was developed from this 
TPS reliability assessment in order to make specific 
recommendations about where best to invest resources 
among numerous potential investment options in order 
to improve one or more of the following benefit 
metrics: 1) the mean value reliability estimate, 2) the 
reliability distribution standard deviation, and/or 3) the 
confidence level associated with the mean value 
reliability estimate and its standard deviation. At the 
heart of the resource allocation process is a cost / 
benefit modeling and assessment process. The chain 
rule must be invoked to correctly define the investment 
merit: 

dBenefit dX 

Benefit Merit = : — * * Numts (4) 

8X dUnit 

Cost Merit = — * Nimits (5) 

dUnit 

Benefit Merit 

Investment Merit = (6) 

Cost Merit 

For each potential investment option, X (typically, an 
uncertainty MV or StDev), a unit of purchase was 
defined. The number of units (Nunits) helps to defines 
both the benefit and cost. The first term in the right 
hand side of Eq. 4 is the probabilistic, physics-based 
sensitivity. The second term in Eq. 4 is the most 
problematic; this term translates a change a MV or 
StDev of an uncertainty into units of purchase. This is 
not well defined and may be highly subjective. 

For many options, the unit of purchase was simply a 
test (either an arc jet or materials property test); the 
cost of a unit of purchase could be determined by SME 
polling. Flowever, some units of purchase are much 
more difficult to define; for example, to improve the 
test reproducibility uncertainty, one could invest in 
better arc jet testing or measurement methods and to 
improve the test validation uncertainty, one could 


invest in improvements to the basic computational 
ablation model. To improve the failure mode 
definition, one might need to invest in hours of 
programmatic discussion to help better define, and 
achieve consensus, on the failure mode(s) of interest. 

It is critical that a credible model of the cost and 
benefit be developed for each investment option being 
considered. Then, having defined credible cost / benefit 
models for each of the possible investment options, the 
individual pieces are assembled together in a system 
resource allocation Bayesian network model; a sample 
resource allocation Bayesian network for eight 
investment options is notionally shown in Fig. 13, with 
one investment option leg highlighted in red. The goal 
of this figure is simply to illustrate the level of 
modeling complexity involved in solving the resource 
allocation problem. The central cyan colored node, 
“Where to Invest?” is the heart of the resource 
allocation problem. The eight nodes immediately 
surrounding this central node (yellow, purple, orange 
and red) are the possible investment options. Each of 
the cyan nodes connected to the investment option 
nodes asks one of the questions “Is Benefit Merit 
High” and “Is Cost Merit High?” (i.e., meaning low 
cost); each of these questions is then broken down 
further, as shown in Fig. 12, to aid in SME evaluation 
of the investment options. A favorable investment 
merit, as defined by Eq. 6, means large “bang for the 
buck”. 

The probabilistic reliability assessment process 
sensitivities are used to properly scale the investment 
merit of each option. This network, when interpreted 
correctly, then allows one to determine specific 
recommendations for investment ranking. Such has 
been done within the scope of this work by using a 
combination of SME cost / benefit assessments and 
physical sensitivities. 

The results of a similar effort for 19 possible 
investment options, subject to many assumption and 
caveats, indicate the most fruitful investment areas are 
computational improvements, arc jet testing 
improvements, more arc jet reproducibility and 
validation test samples, and more failure mode test 
samples. These are the areas of the greatest 

uncertainty and influence within the reliability 
assessment. 

An uncertainty design process has likewise been 
developed and executed based upon the probabilistic 
reliability assessment process sensitivities. In this 
process, the analyst changes the TPS thickness in the 
presence of all the other quantified uncertainties. This 
process allows one to determine the TPS thickness for 
each of the seven body points under consideration, for 


the current vehicle trajectory, such that the TPS meets 
the target reliability statement. 

3. SUMMARY 

Numerous uncertainties for a human-rated TPS system 
have been identified and quantified. Among the 
greatest challenges are improving the reproducibility 
and validation of the arc jet test results, properly 
statistical convergence of data, material property 
correlations, the effects of flow transition upon the 
chosen failure mode, the effects of a reliability cascade 
backward through the process and credible cost / 
benefit modeling to enable improving upon the current 
situation. The various uncertainties have been 
modelled within a physics-based probabilistic 
reliability assessment process and propagated through 
the process to determine a TPS reliability estimate 
distribution and its sensitivities to both the mean value 
and standard of the modelled uncertainties. The 
sensitivities are then used to drive both resource 
allocation and design under uncertainty processes. 
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Fig. 1. Generic synthesis and modeling process. Fig. 4. Generic reliability assessment process. 



Fig. 2. Generic code development process. 
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Fig. 5. One-Dimensional ablation process. 
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Fig. 6. Arc jet testing process sketch. 


Fig. 3. Generic code execution process. 
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Fig. 7 Uncertainties sketch. 



Fig. 10. Reliability estimation sketch, required 
reliability = 50%. 
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Fig. 8. Heat shield body points sketch. 
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Fig. 11. Reliability estimation sketch, required 
reliability = 90%. 
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Fig. 9. Failure mode sketch. 


Generic Cost to Benefit Ratio Evaluation 



Fig. 12. Generic cost / benefit modeling 
Bayesian network sketch. 
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Fig. 13. Generic 8-option resource allocation Bayesian network sketch. 


